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EVALUATION  CRITERIA  FOR  REAI^TIME  SPECIFICATION  LANGUAGES 


1.  Introduction 

Thb  report  proposes  a  set  of  evaluation  criteria  for  languages  designed  to  specify  the 
requirements  of  real-time  systems.  It  is  intended  for  a  reader  who  b  beginning  a  real-time 
development  project  and  considering  a  method  or  language  for  capturing  the  system’s 
requirements.  We  assume  the  reader  b  familiar  with  at  least  some  real-time  specification 
languages  and  with  the  characterbtics  that  dbtingubh  real-time  systems  from  others. 
Specification  languages,  for  the  purpose  of  thb  study,  include  both  highly  formal  languages 
(having  at  least  a  formal  syntax)  as  well  as  informal  ones.  We  include  technical  criteria  we 
believe  may  be  formally  evaluated,  such  as  the  ability  to  verify  timing  properties;  we  also 
include  criteria  of  a  more  subjective  nature,  such  as  readability  and  ease  of  use.  For  each  we 
include  a  Ibt  of  key  questions  that  a  developer  may  use  to  help  evaluate  a  candidate 
language. 

Not  surprbingly,  the  criteria  are  not  unrelated,  although  how  they  affect  each  other 
varies  from  case  to  case.  For  example,  increasing  the  formality  of  a  language  may  increase  or 
decrease  the  readability  of  the  specification.  A  language  with  a  strong  conceptual  construct 
and  high  applicability  to  real-time  systems  probably  produces  a  very  concise  specification, 
which  may  be  easier  to  modify,  but  an  overly  concise  specification  (or  an  overly  verbose  one) 
may  have  very  poor  readability. 

We  do  not  provide  value  rankings  for  the  criteria,  because  the  value  varies  with  pro¬ 
jects.  For  example,  ease  of  learning  would  be  more  important  to  a  project  staffed  with 
unskilled  or  inexperienced  personnel  than  one  with  seasoned  veterans;  sophbticated  support 
toob  may  be  irrelevant  to  a  project  without  the  computing  resources  to  exploit  them. 

We  do  not  at  thb  time  provide  objective  measurement  procedures  for  the  criteria.  For 
some  criteria,  derivation  of  measurements  represents  an  obvious  continuation  of  thb  work 
and  b  beyond  the  scope  of  the  current  effort.  For  others,  it  bn’t  clear  that  finding  an  objec¬ 
tive  measure  b  feasible.  In  any  case,  we  believe  that  the  identification  of  the  criteria  b  useful 
in  its  own  right.  It  should  motivate  the  project  manager  to  think  about  long-term  issues  and 
provide  a  justification  framework  for  choosing  a  particular  language  and  rejecting  others. 

The  evaluation  criteria  are  presented  in  two  sections.  Section  2  suggests  language 
features  that  support  desirable  properties  of  the  finbhed  product — the  requirements 
specification.  Section  3  proposes  language  features  that  facilitate  the  process  by  which  the 
specification  b  produced.  A  brief  summary  appears  in  Section  4.  A  bibliography  and  glos¬ 
sary  relevant  to  real-time  system  specification  conclude  thb  report. 


2.  Product-Oriented  Criteria 

The  product  under  consideration  b  the  requirements  specification.  The  purpose  of  a 
requirements  specification  language  is  to  establbh  a  syntactic  and  semantic  context  in  which 
to  develop  that  product.  The  purpose  of  a  requirements  specification  b  to  define  all  accept¬ 
able  implementations  of  a  system  and  to  specify  any  constraints  on  its  implementation  [Heit- 
meyer  and  McLean  1983j.  We  take  as  axiomatic  that  a  requirements  specification  should  be 
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unambiguous,  complete,  verifiable,  consistent,  easy  to  change,  traceable,  and  usable  during 
development,  operation,  and  maintenance  [ANSI/IEEE  Std  830-1984].  A  specification 
language  must  at  least  permit  such  properties  in  a  requirements  specification.  It  might 
guarantee  them  (by  making  it  impossible  to  produce  a  product  without  them);  it  might  sim¬ 
ply  encourage  them  (by  providing  features  designed  to  ease  their  rendering).  Note  that  a 
language  that  guarantees  a  good  product  need  not  be  the  best  choice;  it  might,  for  instance, 
be  prohibitively  bard  to  use. 

In  the  following  subsections,  we  suggest  evaluation  criteria  related  to  the  production  of 
high-quality  requirements  specifications  in  the  context  of  real-time  systems. 


2.1.  Applicability  to  Real-Time  Systems 

Since  real-time  systems,  by  definition,  must  respond  to  events  under  some  timing  con¬ 
straints,  it  is  imperative  that  the  language  for  rendering  real-time  specifications  be  able  to 
express  such  timing  requirements.  The  existence  of  a  model  for  timing  in  the  requirements 
specification  language,  and  the  notation  for  expressing  timing  constraints  is  the  primary  issue 
that  sets  realrtime  and  non-real-time  specification  languages  apart.  The  timing  model  may 
be  based  upon  either  continuous  or  discrete  time.  Furthermore,  soft  real-time  systems  deal 
with  stochastic  performance  models;  their  requirements  are  written  in  terms  of  a  some 
minimum  number  of  times  that  a  real-time  deadline  must  be  met.  By  contrast,  hard  real¬ 
time  systems  use  deterministic  models;  their  minimum  required  rate  for  satisfying  a  deadline 
is  100%.  Some  ^sterns,  such  as  the  Space  Shuttle,  combine  aspects  of  both  soft  and  hard 
real-time:  a  set  of  primary  or  high-priority  tasks  must  always  meet  their  deadlines,  whereas 
it  is  permissible  for  less  important  tasks  to  fail  to  complete  from  time  to  time. 

It  is  not  sufficient  that  requirements  for  real-time  systems  express  only  an  ordering  of 
events,  system  responses,  etc.;  they  must  also  express  absolute  and  relative  time  intervak 
from  a  fixed  starting  point.  Timing  constraints  should  be  stated  only  in  terms  of  events  that 
are  externally  visible  at  the  system  level.  To  achieve  this  goal  the  model  of  the  system 
environment  embraced  by  the  language  must  be  complete  and  well-defined. 

Some  specification  languages  suitable  for  real-time  systems  may  have  semantics  for 
parallelism,  which  some  real-time  applications  may  need.  The  semantics  of  parallelism  in  the 
specification  language  may  use  either  a  maximal  parallelism  model  or  an  interleaving  model. 
Maximal  parallelism  allows  any  number  of  events  to  occur  simultaneously,  as  in  the  real 
world.  The  interleaving  model,  on  the  other  hand,  forces  simultaneous  events  to  be  sequen- 
tialized  artificially.  Interleaving  is  considered  inadequate  by  some  for  handling  certain  situa¬ 
tions  involving  simultaneity  in  a  meaningful  manner  |Mok  1991).  Others  find  that  it  is  possi¬ 
ble  to  incorporate  time  into  an  interleaving  model  to  represent  real-time  adequately  [OstrofT 
1989). 

Key  questions  about  the  applicability  of  the  language  to  real-time  systems,  then, 
include; 

(1)  Can  the  language  express  absolute  and  relative  timing  constraints? 

(2)  Is  the  language’s  model  of  time  discrete  or  continuous? 

(3)  Can  timing  constraints  be  expressed  only  in  terms  of  events  observable  to  the  system  in 
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(4)  Can  the  language  express  stochastic  requirements  for  deadline  satisfaction? 

(5)  Can  the  language  express  what  is  required  to  occur  if  a  timing  constraint  is  missed? 

(6)  Can  the  language  express  parallelism?  Does  it  use  the  true  or  interleaved  model  of 
parallelism? 

2.2.  T  >re8eiiting  the  Conceptual  Construct 

A  major  concern  in  representing  a  set  of  requirements  is  capturing  the  essential  proper¬ 
ties  or  conceptual  construct  [Brooks  1987]  of  a  system  while  leaving  unspecified  those  details 
that  do  not  affect  validity.  The  specification  should  serve  as  an  abstraction  representing 
exactly  the  set  of  all  valid  implementations,  and  neither  overspecify  (provide  details  that  are 
not  requirements)  nor  underspecify  (omit  details  that  are  requirements).  A  specification 
should  say  what  is  required  of  the  system  and  not  how  that  system  is  implemented,  i.e.,  it 
should  represent  a  “black  box”  with  only  the  externally  observable  behavior  specified  [Parnas 
1979]. 

A  difficulty  in  representing  the  conceptual  construct  is  that  inessential  artifacts  of  a 
specification  may  be  misconstrued  as  part  of  the  conceptual  construct.  Such  is  often  the  case 
with  specifications  that  use  operational  definitions  whose  details  may  be  misinterpreted  as 
design  or  implementation  constraints.  For  example,  a  specification  may  include  parallelism 
as  a  conceptual  construct,  but  it  would  be  a  premature  design-level  decision  to  interpret  this 
as  requiring  either  a  parallel  system  or  a  distributed  system  architecture  (unless  mandated  as 
a  constraint).  Also  undesirable  are  specification  languages  that  use  design-level  concepts  such 
as  data  flow  modeb,  because  the  specifications  resulting  from  such  techniques  usually  imply 
that  a  particular  component  architecture  b  required.  Even  though  the  ideal  conceptual  con¬ 
struct  for  a  system  can  at  best  be  subjectively  evaluated,  it  b  desirable  that  a  specification 
language  support  modeb  and  notations  that  minimize  confusion  about  which  constructs  are 
required  properties  (i.e.,  external  behavior)  and  which  are  artifacts  of  the  specification. 

Legitimate  design  and  implementation  constraints,  which  tend  to  decrease  the  number 
of  potentially  valid  implementations  by  limiting  choices  for  designers  and  implementors,  must 
be  handled  with  care.  Different  notations  may  be  appropriate  for  such  constraints,  since  it  b 
important  that  true  constraints  not  be  confused  with  similar  appearing  constructs  that  are 
only  artifacts  of  the  specification. 

Families  of  systems  arise  anytime  when  a  requirements  specification  leaves  a  choice 
open  to  the  designer  or  implementor.  While  implicit  choices  left  to  the  designer  or  imple¬ 
mentor  give  rise  to  families  of  different  implementations,  we  emphasize  explicit  requirements 
constructs  that  provide  additional  family  concepts: 

•  Nondeterminbm  allows  for  different  choices  based  upon  alternate  behaviors  that  are 
equally  suitable. 

•  Abstract  input  or  output  devices  define  families  of  systems  with  common  functionality 
but  choice  of  hardware  devices. 

•  Generic  system  parameters  (analogous  to  those  of  the  Ada  programming  language)  give 
rise  to  families  of  systems  that  differ  in  the  values  of  those  system  parameters. 

Such  concepts  in  a  specification  tend  to  increase  the  number  of  potentially  valid  implementa¬ 
tions  and  may  make  a  specification  reusable  or  more  easily  modified.  Language  support  for 
families  of  systems  should  be  flexible  enough  to  include  the  full  range  from  narrowly  defined 
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single  systems  with  many*  constraints  to  general  families  of  systems. 

Part  of  a  system’s  conceptual  construct  is  its  operating  environment.  Developing  a 
real-time  system  may  require  extensive  modeling  of  the  external  environment  (such  as  a  tim¬ 
ing  model  for  it)  in  order  to  describe  or  analyze  the  requirements  properly.  For  further  dis¬ 
cussion  of  this  concern  see  [Heitmeyer  and  McLean  1983]. 

Key  questions  about  capturing  a  system’s  conceptual  construct,  then,  include: 

(1)  Does  the  language  lend  itself  to  specifying  only  what  is  required,  or  is  it  burdened  by 
the  need  to  include  irrelevant  detaib  or  other  artifacts  of  using  that  language  in  the 
specification? 

(2)  Does  the  language  facilitate  the  representation  of  program  families  by  allowing  abstrac¬ 
tion  of  parameter  values,  hardware  devices,  and  nondeterminism? 

(3)  Does  the  language  provide  for  modeling  the  system’s  operating  environment? 

2.3.  Formality 

As  with  many  of  these  criteria,  formality  is  a  spectrum  quality  rather  than  an  absolute. 
A  formal  specification  language  has  at  least  a  precise,  rigorously  defined  syntax.  That  means 
that  it  is  possible  to  test  unambiguously  whether  a  specification  is  a  member  of  the  langus^e 
or  not.  In  addition,  some  formal  languages  have  precisely  defined  semantics.  A  specification 
written  in  such  a  language  has  mathematical  properties  that  can  be  analyzed.  More  to  the 
point,  it  can  be  shown  that  any  :^stem  that  meets  that  specification  will  have  certain  proper¬ 
ties.  For  example,  desired  invariants  can  be  derived  for  most  real-time  systems;  an  example 
invariant  is  “the  valve  will  always  be  closed  within  two  seconds  of  detection  of  sensor  tem¬ 
perature  exceeding  212  degrees.”  Proving  that  desired  invariants  are  implied  by  the 
specification  is  an  indispensable  exercise  in  making  sure  that  the  specification  is  valid. 

Formal  specifications  also  have  the  potential  to  be  processed  mechanically;  for  example, 
a  correctness  proof  can  be  checked  automaticsJly,  even  if  the  proof  could  not  be  automati¬ 
cally  derived  [Liskov  and  Berzins  1979}.  Completeness  and  consistency  checks  can  be 
automated  because  there  is  a  formal  definition  of  both.  By  contrast,  natural  language 
specifications  can  be  processed  mechanically  only  at  the  most  superficial  levels,  such  as  sim¬ 
ply  manipulating  various  blocks  of  text.  However,  the  role  of  natural  language  commentary 
should  not  be  overlooked  for  clarifying  the  major  points  of  a  formal  specification  or  providing 
background  and  motivation  for  the  decisions  embodied  by  the  formalisms. 

Formal  specification  languages  can  be  judged  by  the  ease  with  which  their  specifications 
can  be  checked  for  a  range  of  properties.  These  properties  include  completeness,  consistency, 
lack  of  ambiguity,  and  verifiability,  each  of  which  is  discussed  below. 


2.3.1.  Completeness 

A  requirements  specification  is  complete  if  it  has  all  the  information  needed  to  define  at 
least  one  system  that  is  acceptable  to  the  customer.  This  also  includes  support  for  any  attri¬ 
butes  that  contribute  to  completeness,  such  as  robustness — the  ability  to  handle  any  possible 
input  conditions  including  errors. 

The  specification  language  must  tolerate  incompleteness  in  a  given  requirements 
specification  during  development,  although  the  language  must  also  aid  in  detecting 
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incompleteness  so  that  it'can  eventually  be  eliminated.  Certain  constructs  are  needed,  such 
as  TBD’s,  that  allow  for  reasonable  analysis  of  an  incomplete  specification. 

Key  questions; 

(1)  Does  the  language  include  rules  that  define  what  a  complete  requirements  specification 
is? 

(2)  Does  the  language  tolerate  incompleteness  during  development? 

(3)  Does  the  language  facilitate  rapid  identification  of  areas  of  incompleteness  in  the 
specification? 


2.3.2.  Consistency 

Consistency  means  that  no  contradiction  can  be  derived  from  a  set  of  facts.  Incon¬ 
sistent  requirements  specifications  have  no  systems  that  satisfy  all  the  requirements.  A 
requirements  specification  must  be  internally  consistent;  that  is,  no  contradiction  can  be 
deduced  from  within  the  specification.  Specification  languages  should  provide  some  form  of 
internal  consistency  checking.  A  requirements  specification  must  also  be  externally  con¬ 
sistent  with  other  products  of  development,  such  as  the  design,  implementation,  etc.  Tracea¬ 
bility  (see  Section  3.3.)  can  provide  some  support  for  externaJ  consbtency  checking. 

If  formal  reasoning  b  associated  with  a  specification  language,  then  it  b  desirable  that 
the  underlying  formal  logic  system  have  been  shown  to  be  sound.  That  b,  any  theorem 
derived  from  the  specification  must  be  true  in  the  specification  model.  Although  first  order 
predicate  logic  b  sound,  special-purpose  logics  need  to  be  shown  to  be  sound  also  [Berg  et  al. 
1982].  Formal  reasoning  b  necessary  to  precisely  define  and  to  automate  consbtency  check¬ 
ing. 

Key  questions; 

(1)  Does  the  language  contains  rules  from  which  mechanical  self-consbtency  checks  can  be 
derived? 

(2)  Does  the  language  provide  a  means  to  perform  external  consbtency  checks  with  other 
products  of  the  development? 


2.3.3.  Lack  of  Aonbiguity 

Ambiguity  in  a  specification  leads  to  more  than  one  meaning,  when  only  one  b 
intended.  A  specification  langu^e  should  not  have  ambiguity  at  either  the  syntactic  or 
semantic  leveb.  Formal  syntax  and  formal  semantics  are  solutions,  since  formal  constructs 
usually  have  unambiguous  definitions.  However,  even  the  lack  of  ambiguity  found  in  formal 
specifications  may  not  prevent  mbunderstandings,  if  the  reader  does  not  have  the  appropri¬ 
ate  background  and  experience  in  the  langus^e  [Parnas  1979]. 

Key  questions; 

(1)  Does  the  language  have  a  formal  syntax  by  which  syntactic  correctness  of  a 
specification  can  be  unambiguously  judged? 

(2)  Is  the  language  semantically  ambiguous? 
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2.3.4.  Verifiability 

The  quality  of  verifiability  refers  to  the  ability  to  prove  that  some  set  of  properties 
holds  for  a  given  specification.  The  ability  to  verify  properties  of  a  specification  is  one  of  the 
most  important  reasons  for  using  formal  specification  languages. 

Specification  languages  that  can  be  verified  are  usually  built  upon  some  type  of  logic. 
To  make  verification  feasible,  it  must  be  automated.  Complete  automation  of  verification 
requires  a  decidable  language,  i.e.,  one  whose  underlying  logic  is  decidable.  Decidable 
languages,  however,  may  not  be  able  to  express  all  desired  requirements  concepts  that  could 
be  expressed  via  an  undecidable  language.  A  compromise  using  some  restricted  subset  of  an 
undecidable  language  is  generally  sought  in  order  to  provide  the  requisite  expressiveness. 
The  ultimate  goal  is  to  provide  an  efficient  automation  of  proofs,  so  that  specifiers  and 
verifiers  need  not  be  experts  in  proof  techniques. 

Key  questions: 

(1)  Does  the  language  have  a  formal  semantics  that  will  allow  proofs  of  invariant  correct¬ 
ness? 

(2)  Is  the  language  based  on  a  logic  that  has  been  shown  to  be  decidable? 

(3)  What  is  the  computational  complexity  of  the  automatic  proof  techniques,  if  any  are 

provided? 

2.4.  Constructibility,  Expressiveness,  and  Conciseness 

A  language  is  said  to  be  constructible  if  it  is  able  to  express  application  domain  con¬ 
cepts.  A  special  case  of  constructibility  for  real-time  systems  was  discussed  in  Section  2.1. 
Other  concepts  may  include  special  language  constructs  for  vehicle  position  and  attitude, 
chemical  reactions,  fluid  dynamics  quantities,  etc.  The  language  is  said  to  have  high  construc¬ 
tibility  if  (1)  the  way  the  specifiers  think  about  the  problem  domain  is  reflected  in  the  avail¬ 
able  constructs  and  in  the  way  these  constructs  are  combined;  or  (2)  the  specification 
language  is  expressive.  Expressiveness  refers  to  the  ability  to  make  statements  about  many 
types  of  properties,  and  the  ability  to  describe  varied  functionality.  So,  for  example,  even  if 
a  particular  language  is  not  constructible  with  respect  to  both  avionics  and  chemical  process¬ 
ing  domains  because  it  does  not  have  built-in  concepts  specific  to  each  one,  it  may  still  be 
expressive  enough  so  that  specifiers  can  easily  build  suitable  specifications  for  both  domains 
by  using  more  general  (domain-independent)  built-in  features  of  the  language. 

Assuming  that  the  language  is  able  to  express  a  construct  at  all,  as  discussed  above, 
conciseness  refers  to  its  ability  to  express  the  construct  with  a  minimum  of  redundant  or 
irrelevant  information.  Important  features  common  to  many  real-time  systems,  e.g.,  timing 
properties,  should  be  expressible  directly  via  a  small  number  of  primitives,  rather  than 
indirectly  in  terms  of  many  primitives. 

Alternately  viewed,  conciseness  measures  the  lack  of  repetition  (either  actual  or  concep¬ 
tual)  necessary  in  a  document.  The  expressive  primitives  are  powerful  because  they  take  the 
bulk  of  common  information  from  the  specification  and  move  it  into  the  semantics  of  the 
language.  Support  for  some  form  of  abbreviations  (such  as  macros)  can  also  aid  in  concise 
specifications  by  factoring  out  repetitive  parts  of  a  specification.  Avoiding  such  repetition 
also  promotes  consistency  within  the  specification  by  maintaining  a  single  definition  of  a  con¬ 
cept. 
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Key  questions: 

(1)  What  features  of  the  language  are  specifically  relevant  to  the  problem  domain  under 
consideration? 

(2)  What  features  of  the  language  are  relevant  across  application  domains? 

(3)  To  what  degree  must  information  (whether  detailed  requirements,  conceptual  back¬ 
ground,  or  semantic  constructs)  be  repeated  in  a  specification  written  in  the  language 
under  consideration? 

(4)  How  compact  is  the  expression  of  information  in  the  language  under  consideration, 
compared  to  that  of  other  languages? 

2.5.  Scalability 

It  is  important  that  the  specification  language  can  handle  scaling  up  from  small,  toy 
problems  to  production  real-time  systems.  A  major  difficulty  in  scalability  is  that  the  com¬ 
plexity  of  large  systems  increases  nonlinearly  with  the  size  of  the  system  [Brooks  1987],  The 
best  evidence  of  language  scalability  is  the  existence  of  previous  application  of  the  language 
to  large  production-quality  real-time  systems,  along  with  documented  evaluation  of  the 
language’s  tool  and  methodology  support.  Lacking  such  a  posteriori  evidence,  the  following  a 
priori  criteria  can  be  used: 

•  The  language  should  support  vertical  decomposition  of  a  specification  from  the  top  level 
(most  abstract)  through  refinement  to  additional  more  detailed  levels.  Consistency  must 
be  maintained  among  multiple  vertical  levels  that  comprise  a  specification. 

•  At  each  level  of  the  vertical  decomposition,  there  should  be  constructs  for  pso-titioning 
the  specification  into  more  manageable  work  assignments  (horizontal  decomposition). 

Key  questions: 

(1)  Is  there  testimonial  evidence  of  application  of  the  language  and  its  methodology  to  pro¬ 
duction  systems? 

(2)  Does  the  language  support  vertical  or  horizontal  decomposition  of  the  system? 

2.6.  Modifiability 

Modifiability  is  the  quality  that  makes  a  specification  easy  to  change.  Requirements  for 
real-time  systems  will  likely  change  many  times  during  the  evolution  of  a  project  due  to  such 
factors  as  changing  environment  md  changing  customer  needs  under  complex  technological, 
legal,  political,  and  social  pressures  [Brooks  1987].  The  language  must  support  ease  of 
change  throughout  the  [^stem’s  evolution. 

In  general,  readability  factors,  such  as  indices  and  cross-referencing  or  their  automated 
equivalents,  contribute  to  ease  of  change.  Additionally,  structuring  to  facilitate  anticipated 
changes  may  be  beneficial.  However,  structuring  criteria  for  modifiability  may  conflict  with 
those  for  readability  and  scalability,  and  require  a  compromise. 

Key  questions: 

(1)  Does  the  language  support  browsing  facilities  (hardcopy  or  online)  that  facilitate  locat¬ 
ing  related  sections  during  modification? 
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(2)  Does  the  language  support  the  documentation  of  anticipated  changes? 


2.7.  Readability 

Readability  is  a  quality  that  enables  individuals  in  different  roles  (specifiers,  customers, 
users,  verifiers,  and  implementors)  to  understand  a  specification  without  undue  difficulty. 
Each  role  has  its  own  perspectives  and  assumptions,  and  requires  different  educational  back¬ 
grounds,  knowledge,  and  experience.  Those  portions  of  the  specification  relevant  to  each  role 
should  be  clearly  understandable  to  each  person  serving  in  that  role. 

The  structure  of  the  specification  also  affects  readability.  It  is  preferable  that  the 
specification  language  aid  in  separating  normal  processing  from  error  processing.  Including 
indices  and  cross-references  (or  online  retrieval  equivalents)  also  promotes  understanding,  as 
well  as  modifiability. 

Key  questions; 

(1)  Does  the  language  provide  for  the  needs  of  readers  in  different  roles? 

(2)  Can  specifications  be  structured  for  readability  (e.g.  normal  vs.  error  processing)? 

(3)  Does  the  language  support  browsing  (hardcopy  or  online)? 


3.  Process-Oriented  Criteria 

Development  of  a  good  specification  requires  the  basic  processes  of  creation, 
modification,  and  analysis.  The  process  of  creation  should  be  supported  by  a  method  that 
provides  guidance  to  the  specifier.  Analysis  of  a  specification  takes  two  basic  forms. 
Verification  (definition  1  in  Glossary)  and  testing  apply  to  properties  such  as  consistency, 
timing,  security,  and  reliability  that  are  sufficiently  formal  to  permit  objective  evaluation. 
Properties  such  as  readability,  maintainability,  and  suitability  of  the  system  to  customer 
needs  require  a  subjective  evaluation  or  validation.  Modification  and  analysis  must  normally 
be  iterated  until  there  is  agreement  with  the  customer  that  the  specification  is  satisfactory. 

At  later  stages  of  the  life  cycle,  verification  (definition  2)  and  testing  of  designs  and 
implementations  with  respect  to  the  requirements  specification  will  occur.  Traceability,  as  a 
complement  to  analysis,  provides  limited  assurance  that  all  requirements  have  been  covered 
both  during  specification  development  and  at  later  stages. 

For  building  large  real-time  systems,  automation  of  these  processes  in  terms  of  tools 
and  environment  is  an  overriding  concern,  as  the  complexity  of  such  systems  is  generally 
unmanageable  without  tool  support. 


3.1.  Method  for  Specification  Creation  and  Modification 

A  requirements  specification  language  either  implies  or  explicitly  provides  a  method  by 
which  the  language  is  used  to  create  and  modify  a  specification.  The  method  may  consist  of 
a  sequence  of  steps  (i.e.,  suggestions  of  what  to  do  next),  procedures  or  heuristics  for  execut¬ 
ing  each  step,  rules  for  evaluating  the  results,  etc.  Guidance  should  be  in  terms  of  when  to 
apply  various  constructs  of  the  language,  as  well  as  how  to  apply  these  constructs  effectively, 
especially  when  there  may  be  a  choice  of  applicable  constructs.  Finally,  the  guidance  should 
not  be  so  rigid  as  to  encumber  the  creativity  or  productivity  of  the  specifier  [STARTS  1987]. 
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Ease  of  use  of  the ‘method  should  be  evaluated  and  considered.  Ideally,  the  method 
should  require  little  formal  training  and  mastery  of  few  unfamiliar  or  complicated  concepts. 
Of  course,  the  benefits  of  the  method  must  be  weighed  against  its  start-up  cost;  one  might  be 
willing  to  invest  in  a  long  training  course  if  the  method  seemed  likely  to  deliver  significant 
long-term  benefits.  A  valuable  characteristic  of  a  method  is  to  allow  decomposition  of  the 
specification  task  into  small  work  assignments  so  that  a  team  of  specifiers  can  cooperate  and 
at  the  same  time  work  relatively  independently. 

The  maturity  of  the  method,  whether  potential,  rudimentary,  or  fully  mature,  should 
be  an  important  factor  in  evaluating  the  method  associated  with  a  specification  language 
[Zave  1991].  Similarly,  the  level  of  method  support,  ranging  from  no  support  for  a  “bare” 
specification  language  to  a  specification  language  embedded  in  a  methodology  that  addresses 
the  entire  system  life  cycle,  should  also  be  considered.  The  specification  method  should  also 
be  compatible  with  other  methods  used  during  the  system  life  cycle. 

Key  questions; 

(1)  Is  guidance  or  heuristics  provided  for  creating  and  modifying  specifications? 

(2)  How  difficult  or  expensive  is  training  in  the  method? 

(3)  Can  the  method  support  division  of  the  specification  process  into  independent  work 
assignments? 

(4)  How  mature  is  the  method? 

(5)  How  does  the  method  integrate  with  others  in  the  system  life  cycle? 

3.2.  Verification  and  Testing 

A  verification  technique  guarantees  that  a  specification  satisfies  some  property  for  all 
states  of  the  system,  in  contrast  to  testing,  which  can  only  show  the  satisfaction  of  that  pro¬ 
perty  for  some  states.  In  evaluating  a  specification  language,  one  should  consider  which 
verification  and  test  procedures  have  been  established,  and  the  level  of  support  via  methods 
and  tools  to  aid  in  such  analyses. 

The  primary  concern  is  verification  and  testing  techniques  that  apply  directly  to  the 
requirements  specification;  for  example,  a  formal  specification  may  lead  to  inexpensive 
automatic  test  case  generation.  Verification  or  testing  of  designs  and  implementations  versus 
a  specification  will  occur  at  later  stages  of  system  development,  e.g.,  correctness  (definition  1) 
of  an  implementation  with  respect  to  its  specification.  Verification  and  testing  techniques  at 
the  design  and  implementation  levels  should  also  be  factors  in  evaluating  a  specification 
language. 

Key  questions; 

(1)  Which  verification  and  test  procedures  are  available  at  requirements  level? 

(2)  Which  verification  and  test  techniques  are  available  during  later  design  and  implemen¬ 
tation? 

3.3.  Traceability 

Traceability  provides  for  relating  objects  at  one  stage  of  development  to  objects  at  the 
next  stage.  Tracing  between  two  development  steps  provides  two  major  forms  of  compliancy 
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checks;  (1)  coverage  of  the  former  system  by  the  latter  (each  former  object  corresponds  to 
one  or  more  latter  objects)  ,  and  (2)  necessity  of  the  latter  objects  (each  latter  object 
corresponds  to  one  or  more  former  objects). 

For  requirements  specification  there  are  two  important  forms  of  traceability  that 
should  be  compatible  with  a  specification  language,  with  traceability  links  both  forward  (link¬ 
ing  the  requirements  specification  to  another  work  product)  and  backward  (linking  that  work 
product  back  to  the  requirements  specification)  [Davis  1990]: 

•  The  requirements  specification  language  should  provide  a  means  of  tracing  each  require¬ 
ment  to  its  maniiestation  in  the  design  and  implementation  as  a  rudimentary  form  of 
verification  or  testing.  This  should  be  supplemented  by  verification  and  testing  for 
more  complete  analysis. 

•  The  requirements  specification  language  should  provide  a  means  of  tracing  each  require¬ 
ment  to  its  informal  expression  by  the  customer  as  an  aid  in  validation. 

Key  questions: 

(1)  Is  traceability  from  informal  customer  requirements  to  the  requirements  specification 
supported? 

(2)  Is  tracing  from  the  requirements  specification  to  design  or  implementation  supported? 

3.4.  VaJidation 

Validation  that  a  specification  satisfies  the  customer’s  needs  is  at  present  an  informal 
process  involving  the  specifier,  the  customer,  and  the  requirements  specification.  Language 
support  for  specification  properties,  such  as  readability  and  traceability,  can  help  in  this  pro¬ 
cess,  as  well  as  support  for  techniques  such  as  prototyping,  scenarios,  and  specification  execu¬ 
tion  (e.g.,  step  by  step  execution  of  a  STATEMATE  specification  that  provides  visual 
highlighting  of  the  currently  active  state  [Harel  et  al.  1990]).  Verification  or  testing  of  pro¬ 
perties,  such  as  timing  and  safety,  provide  additional  input  to  the  validation  process. 

Key  questions: 

(1)  Which  properties  (e.g.,  readability)  related  to  the  language  aid  in  validation? 

(2)  Which  techniques  (e.g.,  prototyping,  specification  execution)  related  to  the  language  aid 
in  validation? 

3.5.  Tools  and  Environment 

Manual  introduction  of  methods,  analyses,  and  traceability  can  only  provide  limited 
support.  To  scale  up  to  production  systems  ultimately  requires  well-integrated  tool  support, 
as  could  be  provided  by  a  CASE  environment,  simply  to  cope  with  the  amount  of  data  and 
its  different  uses  by  the  people  involved  in  the  requirements  specification  process. 

Various  quality  factors  will  affect  the  acceptance  of  automation  in  place  of  manual  tech¬ 
niques.  Took  and  a  supporting  environment  must  be  cost-effective.  Tools  should  be  robust, 
easy  to  learn,  and  easy  to  use.  Furthermore,  simple  tools  (such  as  syntax-checking  editors) 
may  suffice  when  the  major  concern  is  with  recording  the  specification  rather  than  extensive 
analyses  [Place  et  al.  1990]. 
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Some  basic  features  of  tools  and  environments  that  should  be  integrated  with  a 

specification  language  and  its  related  method  and  analyses  include  the  following: 

•  The  environment  should  provide  a  common  repository  for  all  requirements  and  various 
useful  relationships  among  different  types  of  information.  Special-purpose  editors  (e.g., 
structured,  syntax-directed,  graphical)  should  provide  for  the  efficient  and  correct  entry 
of  requirements  data  in  multi-user  mode. 

•  Open,  non-proprietary  data  formats  and  interfaces  should  be  standardized  to  promote 
interoperability  among  tools. 

•  The  environment  should  provide  configuration  management  and  version  control  for  the 
various  work  products  of  the  requirements  process. 

•  The  organization  of  the  environment  and  the  requirements  data  should  support  various 
analysis  tools. 

Key  questions; 

(1)  Are  the  available  tools  supporting  the  langu^e  cost-effective,  robust,  easy  to  learn,  and 
easy  to  use? 

(2)  Are  these  tools  interoperable  with  the  development  environment? 


4.  Summary 

We  have  developed  a  set  of  general  evaluation  criteria  for  real-time  requirements 
specification  languages.  These  criteria  cover  important  properties  of  a  specification  (applicar 
bility  to  real-time  systems,  capturing  the  conceptual  construct,  etc.)  that  should  be  supported 
by  a  specification  language,  as  well  as  techniques  for  analyzing  those  properties  (verification, 
traceability,  etc.).  These  general  criteria  are  intended  as  a  guide  to  the  development  of  more 
detailed  criteria  during  actual  evaluations  of  specification  languages. 

We  close  by  reminding  the  reader  that  choice  of  language  plays  only  a  limited  role  in 
the  success  of  a  development  effort.  Although  a  language  may  facilitate  sound  engineering 
practices,  it  is  still  incumbent  on  the  engineering  staff  and  project  management  to  enforce 
those  practices. 
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GLOSSARY 


acceptable 


abstraction 


ambiguity 


analysis 


completeness 


conceptual  construct 


conciseness 

consistency 


constraint 


Satisfying  the  customer’s  “real”  requirements  for  a  system.  The 
customer’s  requirements  may  not  be  the  same  as  those  actually 
in  the  requirements  specification,  since  the  requirements 
specification  may  not  correctly  capture  the  “real”  requirements. 
Synonym  for  valid. 

The  process  and  product  of  choosing  only  certain  attributes 
from  many  that  exist  of  an  object  or  concept.  The  chosen  attri¬ 
butes  are  important  with  respect  to  some  goal. 

Lack  of  precision  (fuzziness),  which  allows  multiple  interpreta¬ 
tions  of  a  given  aspect  of  a  specification,  at  least  one  of  which 
would  lead  to  an  unacceptable  implementation. 

1.  Process  of  testing  or  verifying  that  a  system  has  certain  pro¬ 
perties,  for  example,  those  described  in  its  requirements 
specification.  2.  Process  of  determining  if  the  requirements  in 
the  specification  are  consistent  with  the  “real”  customer  require¬ 
ments  (i.e.,  validation). 

Quality  that  all  relevant  information  for  developing  an  accept¬ 
able  implementation  has  been  included  in  the  specifications. 
With  incomplete  specifications  it  is  possible  to  develop  at  least 
one  unacceptable  implementation. 

The  essential  properties  of  a  specified  system.  This  represents 
what  is  needed  to  specify  the  valid  implementations  of  the 
system — no  more  (overspecification)  and  no  less 
(underspecification). 

Compact  expression  of  a  concept. 

1.  (Internal  consistency)  Quality  of  a  requirements  specification 
such  that  there  are  no  contradictions  or  conflicts  among  any  of 
its  parts.  2.  (External  consistency)  Quality  of  a  requirements 
specification  that  there  are  no  contradictions  or  conflicts 
between  the  specification  and  another  product  of  the  develop¬ 
ment  process. 

Any  decision  that  limits  the  set  of  valid  implementations  of  a 
system.  These  may  be  hardware  constraints  (e.g.,  computer  X 
must  be  used),  software  constraints  (e.g.,  database  package  D 
must  be  used),  or  non-functional  properties  such  as  performance 
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and  reliability. 


correctness 


customer 


distributed  system 


environment 


formal  specification 


formal  verification 


hard  real-time  system 


informal  specification 


1.  Quality  of  having  consistency  between  the  requirements 
specification  and  any  of  the  other  development  products  (for 
example,  design  specification  or  the  implementation).  2.  Quality 
of  having  consistency  between  the  customer’s  “real”  require¬ 
ments  for  a  system  and  the  final  system. 

The  person(s)  who  contracts  and  pays  for  the  development  of  a 
system.  The  customer  usually  (but  not  necessarily)  defines  the 
system  requirements  (ANSI/IEEE  Std  830-1984]. 

A  system  operating  with  multiple  processors,  and  one  or  more  of 
those  processors  shares  no  common  memory  with  the  others. 
This  makes  message  passing  for  communication  a  requirement. 

1.  (External)  environment:  The  external  conditions  and  inter¬ 
faces  under  which  a  system  operates.  2.  (Software  engineering) 
environment  (SEE):  The  collection  of  computers,  support 
software,  procedures  and  facilities  that  make  the  tools  and 
methodologies  used  by  software  developers  easily  available 
[Thayer  and  Thayer  1990). 

A  formal  specification  is  one  that  has  an  effective  procedure  to 
tell  whether  a  specification  has  a  particular  property  of  interest 
[Gunter  1991].  This  type  of  specification  tends  to  be  mathemat¬ 
ically  oriented,  and  it  requires  more  knowledge  and  experience 
than  the  informal  type  to  understand.  Informal  explanatory 
comments  may  elucidate  formal  specifications. 

Verification  in  which  the  proof  could  be  recognized  mechanically 
to  be  a  proof,  regardless  of  whether  the  proof  was  developed  by 
hand  or  (partially)  automatically  generated.  (Also  see 
verification.) 

System  in  which  deadlines  for  critical  tasks  (e.g.,  start  or  com¬ 
pletion  times)  must  be  met  to  prevent  some  catastrophe,  or  to 
reduce  the  probability  that  a  catastrophe  would  result. 

A  specification  that  is  written  largely  using  natural  language 
rather  than  formal  mathematical  notations.  The  syntax  and 
notation  may  be  largely  ad  hoc,  rather  than  consistent  in  the 
manner  of  formal  specifications.  An  informal  specification  may 
contain  free-form  commentary  as  part  of  the  specification  itself 
(as  contrasted  to  informal  explanatory  comments  used  with  for¬ 
mal  specifications).  An  informal  specification  is  amenable  to 
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very  limited  machine  manipulation. 


informal  verification 


maintenance 


model 


nondeterminism 


notation 


overspecify 


parallel  system 


precise 


real-time  system 


Verification  in  which  the  proof  cannot  be  recognized  mechani¬ 
cally  to  be  a  proof.  This  type  of  verification  tends  to  be  ad  hoc 
and  to  be  expressed  in  natural  language,  or  informal  notations 
(e.g.,  ones  invented  ad  hoc,  or  mixtures  of  various  notations). 
(Also  see  verification.) 

The  process  of  fixing  errors  encountered  in  a  system  once  it  is  in 
operational  use,  or  changing  it  to  satisfy  new  requirements. 

An  abstraction  that  includes  all  essential  properties  of  the  pro¬ 
cess  or  object  being  modeled,  but  does  not  include  any  irrelevant 
properties.  Relevance  is  determined  by  how  the  model  will  be 
used.  A  specification  is  a  model  of  the  system  to  be  developed. 

Property  that  an  observer  cannot  tell  which  of  several  behaviors 
should  be  chosen,  since  more  than  one  is  acceptable.  It  can  be  a 
desirable  characteristic  in  a  specification.  In  addition,  however, 
repeatability  may  also  be  desired  in  some  subset  of  these  cases, 
so  that  once  a  particular  behavior  is  chosen  later  in  the  develop¬ 
ment  process,  it  may  be  a  requirement  to  use  that  one  alone 
wherever  this  nondeterministic  requirement  appears  [Parnas  and 
Wang  1989]. 

The  means  of  expressing  the  structure  (i.e.,  syntax)  of  a  given 
language  unit  (e.g.,  sentences  in  English,  or  propositions  in  logic) 
via  the  composition  of  symbols.  The  notation  for  expressing 
syntax  is  not  the  same  as  the  syntax. 

1.  To  put  extra  information  into  the  specification  of  a  software  sys¬ 
tem  to  the  extent  that  it  excludes  at  least  one  acceptable  implemen¬ 
tation  (due  to  the  extra  information  being  inconsistent,  being  design 
information  not  appropriate  to  describing  the  externally  visible 
behavior,  etc.).  2.  To  put  undesirably  redundant  information  into  the 
specification  of  a  software  system  [Place  et  al.  1990). 

A  system  with  multiple  processors  that  communicate  via  shared 
memory. 

Well-defined.  Precision  is  a  major  requirement  for  automated  pro¬ 
cessing. 

Any  system  that  must  operate  under  some  form  of  timing  con¬ 
straints.  (See  soft  and  hard  real-time  systems). 
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requirements  specification* 

Product  that  defines  the  acceptable  implementations  for  a  software 
system  subject  to  constraints  (which  may  include  performance, 
operating,  interface,  and  economic  constraints)  [Heitmeyer  and 
McLean  1983,  Roman  1985].  These  constraints  should  be  minimal  so 
that  no  useful  implementations  are  precluded  [Zave  and  Yeh  1981]. 

semantics  The  meaning  of  a  construct  as  opposed  to  its  syntax. 

simulation  Process  of  testing  important  system  properties  by  executing  a  model 

of  the  proposed  system.  The  model  may  have  artificial  (simulated) 
components  for  any  of  the  computer  hardware,  environment,  or 
software  functionality  [Shooman  1983]. 

soft  real-time  system  A  real-time  system  that  has  stochastic  (rather  than  deterministic) 

timing  constraints. 

specification  Description  of  the  essential  properties  of  an  object  (system,  software, 

program,  etc.).  Specifications  may  be  informal  or  formal. 

specification  language  Any  method  providing  the  basic  concepts  and  relations  for  expressing 

specifications.  Specification  languages  include  formal  languages  as 
well  as  less  formal  constructs. 

^ntax  The  structure  of  a  construct  as  implied  by  the  rules  for  comfxjsing  it 

from  subcomponents  (as  opposed  to  its  semantics).  The  syntax  rules 
may  manifest  themselves  through  different  notations. 

testing  Any  process  (e.g.,  regression  testing)  for  establishing  confidence  in 

the  truth  of  some  property  of  a  specification,  design,  implementation, 
etc.  by  checking  or  executing  some  subset  of  the  total  possible  out¬ 
comes.  In  contrast  to  verification,  testing  can  never  absolutely  verify 
(definition  1)  some  property  (except  exhaustive  testing  of  all  possibili¬ 
ties). 

traceability  Identification  and  recording  of  the  links  between  requirements  and 

the  manifestation  of  those  requirements  in  other  products  of  the 
software  development  process  (informal  customer  requirements, 
design,  implementation,  test  plans,  etc.) 

underspecify  To  specify  without  enough  detail  (i.e.,  incompletely,  ambiguously, 

etc.)  such  that  the  specification  admits  at  least  one  unacceptable 
implementation  [Place  et  al.  1990]. 
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user 


Person(s)  who  interacts  directly  with  a  system.  Users  and  customers 
are  not  necessarily  the  same  people  [ANSI/IEEE  Std  830-1984]. 


valid 

validation 

verification 


Synonym  for  acceptable. 

Process  of  determining  that  a  software  system  satisfies  the 
customer’s  needs.  “Am  I  building  the  right  product  [Boehm  1984]?” 

1.  Process  of  proving  that  a  sp>ecification  satisfies  some  property  (for 
all  situations).  2.  Process  of  determining  if  products  of  one  phase  of 
software  development  satisfy  requirements  of  previous  phase.  “Am  I 
building  the  product  right  (Boehm  1984]?” 
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